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ABSTRACT 

Digital globes are new technologies increasingly used in informal and formal education to display global datasets and show 
connections among Earth systems. But how effective are digital globes in advancing public literacy in Earth system science? 
We addressed this question by developing new content for digital globes with the intent to educate and excite the public about 
biological and geophysical processes and exploration in the deep ocean. We developed the content in tight collaboration with 
scientists, educators, and graphic artists. We used global datasets, including a new dataset for locations of deep-sea vents, and 
imagery from deep-diving vehicles to create two narratives. Life Without Sunlight and Smoke and Fire Underwater, each 
targeting a set of Earth Science and ocean literacy principles. Here, we evaluate perceived learning outcomes for the narratives 
delivered as movies and as live, docent-led presentations with the room-sized Science On a Sphere®. Both narratives led to 
median responses of "Some" to "Quite a bit" of perceived learning per literacy principle. Perceived learning outcomes were 
greatest for adult lifelong learners, followed closely by students (under age 18); adult educators generally reported less 
learning. We found no significant differences in the effectiveness of our movies compared to docent-led presentations for any 
of the literacy principles, and all presentations led to "Quite a bit" of excitement for all viewers. Our evaluation provides an 
initial indication that digital globes can be effective in teaching the scientific literacy principles, and we make 
recommendations for additional assessment of digital globes as educational platforms. © 2015 National Association of 
Geoscience Teachers. [DOI: 10.5408/14-067.1] 
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INTRODUCTION 

Spherical display systems, also known as digital globes, 
are new technologies that can inspire students and public 
audiences to learn about Earth system processes. A review of 
research on data visualization experiences including digital 
globes indicated "a variety of positive impacts on learning" 
(Goldman et al., 2010). Digital globes can be used to display 
a growing variety of global datasets—from near-real-time 
earthquakes in the context of Earth's plate boundaries, to 
changes in primary productivity over seasonal cycles on land 
and in the ocean, to hurricane tracks, animal migrations, and 
more. Digital globes also are growing in popularity as unique 
platforms on which to screen films (e.g., Starobin, 2006). It is 
likely not just the display of datasets in a global context but 
also the construction of science stories, navigating the 
datasets, that leads to new learning (Klassen, 2009). For 
example, the importance of narrative accompanying geo¬ 
spatial visualization has been recognized for public literacy 
in climate science (Niepold et al., 2008; Schollaert Uz et al., 
2014). 

We were interested in using digital globes to advance 
public literacy in Earth system science—linking the litho¬ 
sphere, hydrosphere, atmosphere, and biosphere. In partic¬ 


Received 24 November 2014; revised 6 August 2015; accepted 2 September 2015; 
published online 14 December 2015. 

HVoods Hole Oceanographic Institution, Woods Hole, Massachusetts 02543, 
USA 

2 M&E Professional Teacher Consultants, Wareham, Massachusetts 02571, 
USA 

3 Ocean Explorium, New Bedford, Massachusetts 02740, USA 
“Author to whom correspondence should be addressed. Electronic mail: 
stace@whoi.edu. Tel.: 508-289-3536. Fax: 508-457-2134 


ular, we were interested in developing content for informal 
educational settings that would target ocean literacy 
principles (OLPs; Ocean Literacy, 2005; Strang et al., 2007; 
Schubel et al., 2009). Studies of public ocean literacy often 
focus on the coastal marine environment (e.g.. Steel et al., 
2005), but we were interested in public knowledge about the 
deep ocean and connections to the deep Earth. Public 
audiences have been captivated by imagery of deep-sea 
hydrothermal vents, also known as hot springs at the 
seafloor, including seafloor eruptions and unique ecosystems 
thriving in an otherwise cold, dark ocean. Can we connect 
such imagery with global datasets to enable people to 
understand the global context and significance of vents in 
the world ocean and in the dynamic processes of Earth? 

We developed a partnership between the Woods Hole 
Oceanographic Institution and the Ocean Explorium in New 
Bedford, MA, to create content for spherical display systems, 
including Science On a Sphere® (SOS). SOS is a room-sized 
(1.7-m-diameter) digital globe that was developed by the 
National Oceanic and Atmospheric Administration (NOAA) 
for visualization of global datasets "as if the viewer were 
looking at the Earth from outer space" (Albers et al., 2005). 
NOAA's online SOS Data Catalog includes almost 500 
datasets categorized to air, land, water, space and more 
(NOAA, 2015). As of fall 2014, the Ocean Explorium was 
one of more than 100 science museums in 20 countries 
around the world that host an SOS (Kramer, 2014). We 
created datasets for SOS, including locations of Earth's 
known deep-sea hydrothermal vents from the InterRidge 
Vents Database (Beaulieu et al., 2013), and six site-specific 
movies for vents visited by deep-sea vehicles. Our project 
was called the Global Viewport to Deep-Sea Vents in honor 
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of the launching of the new human-occupied vehicle Alvin 
in 2014, with its improved viewports for scientific observa¬ 
tions. 

We linked our new datasets with other datasets from the 
SOS Data Catalog to create science stories with the intent to 
educate and excite the public about biological and geophys¬ 
ical processes and exploration in the deep ocean. Ultimately, 
we developed two educational narratives, Life Without 
Sunlight (LWS) and Smoke and Fire Underwater (SFU)— 
each focusing on a different set of OLPs (Ocean Literacy, 
2005) and Earth Science literacy principles (ESLPs; Wyses- 
sion et al., 2010, 2012). We delivered the two narratives to 
public audiences using the SOS at the Ocean Explorium as 
two types of presentations: either as a movie or as a live, 
docent-led, interactive presentation. The live, docent-led 
presentations used the same datasets and additional site- 
specific video imagery, expanding on the script of the 
narrated compilation movies. Later, we describe how we 
evaluated the effectiveness of the two narratives and two 
types of presentations with respect to the targeted scientific 
literacy principles and level of engagement of the audience. 
Our study is one of the first to examine the effectiveness of a 
narrated movie as a stand-alone presentation on the SOS. 

We were also interested in knowing whether our new 
content for the SOS was as effective for students as for 
adults. The SOS cross-site summative evaluation study was 
mainly designed for adults over age 18 (Goldman et al., 
2010); thus, our data would fill a gap in understanding how 
public audiences of all ages experience the SOS. In a report 
prepared for the NOAA Education Program, Tran (2009) 
noted challenges for public understanding of complex 
systems, especially evident for "students and novices" 
compared to "professionals and scientists." In particular, 
students "tend to miss the interconnectedness and complex 
causal relationships within and among systems" (p. 25 in 
Tran, 2009). The National Academies' Committee on 
Learning Science in Informal Environments noted that 
programs may need to be tailored to age groups, distin¬ 
guishing "children and youth" from "adults, including K-12 
teachers" (Bell et al., 2009). In our study, we distinguished 
students (under age 18), adult lifelong learners, and adult 
educators to address these differences and to fulfill goals of 
the U.S. National Science Foundation, including "preparing 
the geoscience workforce of the future," "life-long learning 
in... informal educational settings," and educators' use of 
"the big ideas" in ocean literacy and Earth Science literacy 
frameworks (p. 10-11 in National Science Foundation 
Advisory Committee for Geosciences, 2012). 

DATASETS AND TWO STORIES FOR SOS 

We conducted a front-end evaluation at our respective 
institutions and at the 2012 SOS Users Workshop to gauge 
public knowledge of deep-sea vents and to aid our choice of 
literacy principles. In particular, we focused on concepts 
recognized as important for public literacy of Earth system 
science: productivity inclusive of marine photosynthesis and 
chemosynthesis, biodiversity inclusive of marine ecosystems, 
geography inclusive of seafloor bathymetry, and plate 
tectonics inclusive of seafloor spreading (Strang et al., 
2007). For example, previous studies indicated that under¬ 
graduate students, even when enrolled in Earth Science 
courses, lack understanding of volcanic systems and plate 
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tectonics (Parham et al., 2010). We identified a subset of 
OLPs (Ocean Literacy, 2005) and ESLPs (Wysession et al., 
2010) to target specifically as we developed our new content. 

We developed our new content through a process 
similar to an iterative software development process (Fox 
and McGuinness, 2008) in which our use case was to 
develop an educational package for spherical display systems 
that would highlight deep-sea vents while linking concepts 
to advance public literacy in Earth system science. The use 
case process is essentially a cycle in which a prototype is 
rapidly developed and evaluated (through formative evalu¬ 
ation) with respect to specific goals. Our design process 
involved tight collaboration in a small team with diverse 
skills, including a lead scientist, educators at both institu¬ 
tions, graphic artists, and a professional evaluator. To scope 
the data and related resources available for our project, we 
started by assessing the available high-definition imagery 
from deep-sea vehicles, georeferenced the imagery to 
locations of hydrothermal vent fields in the InterRidge Vents 
Database (Beaulieu et al., 2013), and determined other 
datasets in the SOS Data Catalog that were relevant to our 
goals. The design specifications for our science stories 
linking the datasets bad to take into account the limited 
time for museum visitors, their potential lack of background 
knowledge to interpret data visualizations, and visual impact 
(e.g., Ma et al., 2012a). Similar to work in the film industry, 
we used storyboarding to aid the development of our 
compilation movies, which include datasets and other 
geoanimations, additional video imagery, and sound (Star- 
obin, 2006; Riedl and Wintner, 2013). For docent-led 
presentations we adopted new features (e.g., layering and 
annotation) in SOS Version 4. The collaboration entailed 
three iterations of the use case cycle in which the latest 
prototype was evaluated in person by the project team at the 
SOS at the Ocean Explorium. Collaborating in a small team 
with mixed skills, each had a role to guide the prototype 
toward a specific goal (e.g., scientist for choosing datasets, 
educators for teaching literacy principles, graphic artists for 
visual impact, and an evaluator to guide our formative 
evaluation). 

Our final product is an educational package that 
includes datasets, movies, scripts, and playlists that incor¬ 
porate other SOS datasets. Our datasets include a static and 
an animated version of the InterRidge Vents Database 
showing the discoveries of deep-sea vents through time 
since 1977. Movies include six site-specific movies for deep- 
sea vents (Axial Seamount, Galapagos Rift, Loihi, Mariana 
Back-Arc, Mid-Cayman Rise, and Mata volcanoes) and two 
narrated compilation movies that interweave global datasets 
with selected portions of the site-specific movies. The full 
package is available for download at the Woods Hole Open 
Access Server (Beaulieu et al., 2014), and educational 
compilation movies were rendered for posting on YouTube 
(Woods Hole Oceanographic Institution, 2014). The datasets 
and movies formatted specifically for the SOS are available 
from the NOAA SOS Data Catalog (NOAA, 2015). 

Ultimately, we constructed two narratives, each high¬ 
lighting three deep-sea vents, and each targeting a different 
set of three literacy principles (Fig. 1). Our two narratives, 
LWS and SFU, were produced as the two compilation 
movies, with content and scripts matched to live, interactive, 
docent-led presentations. We aimed for less than 5 min per 
movie, given previous research indicating the amount of 
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Life Without Sunlight 


Smoke and Fire Underwater 


Biology 


OLP 5.g.: There are deep ocean ecosystems 
that are independent of energy from sunlight 
and photosynthetic organisms. Hydrothermal 
vents rely only on chemical energy and 
chemosynthetic organisms to support life. 


ESLP 6.9.: Life occupies a wide range of 
Earth's environments, including extreme 
environments at seafloor vents where hot 
fluids escape from the oceanic crust. 




Earth 

science 


ESLP 3.2: Earth is a complex system of 
interacting rock, water, air, and life. All Earth 
processes are the result of energy and mass 
moving between Earth's systems, including 
Earth's interior. 




ESLP 4.5.: Many active geologic processes 
occur at plate boundaries. Plate interactions 
affect the locations of volcanoes and the 
distribution of resources and living organisms. 


Exploration 



OLP 7.a.: The ocean is the last and largest 
unexplored place on Earth. This is the great 
frontier for the next generation’s explorers 
and researchers. 



OLP 7.d.: New technologies, sensors, and 
tools such as subsea observatories and 
unmanned submersibles are expanding our 
ability to explore the ocean. 


FIGURE 1: Screen grabs from our two narratives and targeted OLPs or ESLPs verbatim from our survey instrument. 
(Color for this figure can be found in the online version of this paper.) 


time that the public interacts with stand-alone SOS exhibits 
(Mitchell et al., 2012), and about 20 min for a docent-led 
presentation. The docent-led presentations allow more time 
to play each site-specific movie in full and to interact with 
the audience. LWS dives beneath the sunlit ocean to the 
darkness of deep-sea vents, where food webs are fueled by 
chemosynthesis, and specifically targets OLP 5.g: "There are 


deep ocean ecosystems that are independent of energy from 
sunlight and photosynthetic organisms." In addition, LWS 
targets ESLP Big idea 3: "Earth is a complex system of 
interacting rock, water, air, and life," previously identified by 
geoscience educators as an important integrated concept for 
Earth system science literacy (Ladue and Clark, 2012). SFU 
asks whether the viewer knows that there are volcanoes in 
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TABLE I: Residence of survey participants. 


City or Town 

Students 

Lifelong Learners 

Educators 

New Bedford 

29% 

40% 

28% 

Fall River 

17% 

5% 

9% 

Town on Cape Cod 

10% 

5% 

23% 

Other town in southeastern Massachusetts 

35% 

40% 

30% 

Outside of southeastern Massachusetts 

8% 

12% 

9% 


the deep sea with vents spewing hot water and specifically 
targets ESLP 4.5: "Many active geologic processes occur at 
plate boundaries." Both educational narratives integrate a 
number of other datasets available for the SOS, including 
bathymetry, Volcano Locations Globally, and Age of the 
Seafloor. 

We use the term "narrative" to clarify that our two 
"stories" were constructed carefully with the intent for the 
audience to learn from and be engaged with the materials. 
Our definition of narrative is relatively aligned with p. 539 in 
Norris et al. (2005), in that we have a "sequence of events 
about a unified subject.. .connected so that individual events 
can be seen in the perspective of others"; however, our 
sequence of events is more in space than in time (although 
LWS starts with the first discovery of deep-sea vents). We do 
not employ characters in our stories, per se; however, Norris 
et al. (2005) point to a more general definition of characters 
that includes entities, which in our case are the deep-sea 
vents highlighted in each story. We also considered other 
elements of narratives as outlined by Norris et al. (2005) with 
the addition of "The effect of the untold" (p. 402 in Klassen, 
2009), important because of the limited amount of time to 
provide a sphere story. 

METHODS FOR SUMMATIVE EVALUATION 

For our summative evaluation of the two narratives for 
the SOS, we focused on two impact categories for informal 
science education: (1) "Awareness, knowledge or under¬ 
standing (of)" and (2) "Engagement or interest (in)" (p. 11 in 
Friedman, 2008). For learning outcomes, we targeted the 
three scientific literacy principles per narrative (Fig. 1), 
categorized later as the biology, Earth Science, and explora¬ 
tion principles. For engagement, we chose the concept 
"excited" from the positive and negative affect schedule 
(Watson et al., 1988). 

We conducted a posttest-only experimental design with 
self-reporting of knowledge gained and level of engagement. 
Our choice of assessment via perceived learning was in part 
because of our posttest-only design. Our survey instrument 
consisted of a questionnaire with seven questions: three 
demographic, three related to literacy principles, and one 
related to engagement, plus an additional question for 
professional educators (Supp. File 1, which can be found 
online at http://dx.doi.org/10.5408/14-067s2). To reduce the 
time needed to complete the survey (and increase the 
likelihood of participation in it), we limited the number of 
questions on the survey instrument. To reduce the amount of 
text in the survey instrument, we used a subset of the words 
in each literacy principle (we paraphrased one principle, 
ESLP 3.2). The survey instrument was refined by testing with 
staff at the Ocean Explorium to improve clarity and ensure 


the time to complete the survey was less than 5 min. The 
survey was provided either online using eSurveysPro 
software (Outside Software, Inc.) or as a paper hard copy. 
For the online surveys, we installed a kiosk outside the 
entrance to the SOS auditorium, and we provided several 
iPads on site and posted a quick response code for 
participants to use their smart phones to complete the 
survey after the presentation. All surveys were anonymous. 

Demographic questions were mandatory; the first had 
three options as categories only to determine whether the 
respondent was a student (under 18 years of age) or adult 
(age 18 or older), with adults selecting "lifelong learner" or 
"educator." The second demographic question determined 
where the respondent lived (categories in Table I). Partic¬ 
ularly for the students, another goal of this project was to 
engage underrepresented students from schools in the 
Massachusetts South Coast region, which includes the cities 
of New Bedford, Fall River, and 12 surrounding towns. We 
used the answer to the residence question as a proxy for 
underrepresented populations. For example, New Bedford 
has a diverse population, with greater proportion of 
"Hispanic or Latino," "American Indian," and "Two or 
more races" than average in Massachusetts (2010 data from 
U.S. Census Bureau, 2014). In New Bedford, 37% of the 
residents speak a language other than English at home 
(2008-2012 data from U.S. Census Bureau, 2014). In 2013- 
2014, 76% of the students in public schools in New Bedford 
and 78% in Fall River were classified as low income 
compared to 38% for the state (Massachusetts Department 
of Elementary and Secondary Education, 2014). Although 
census geography may serve as a proxy for race or ethnicity, 
we recognize that it is not as strong a predictor as when 
combined with additional data (e.g., Elliot et al., 2009); 
however, this single question also satisfied the museum's 
interest in where its visitors were coming from. The third 
demographic question determined the respondent's level of 
educational attainment (categories in Table II). High school 
graduation rates of 59% for New Bedford and 73% for Fall 
River are low compared to the state rate of 85% (2013 data; 
Massachusetts Department of Elementary and Secondary 
Education, 2014). The proportion of the population in New 
Bedford with a bachelor's (or higher) degree is just 15%, less 
than half the average in Massachusetts (age 25+ 2008-2012 
data from U.S. Census Bureau, 2014). 

For self-reporting of knowledge gained, i.e., perceived 
learning outcomes, we used the same structure for each 
question: We wrote out the literacy principle (listed in Fig. 1), 
asked respondents to indicate whether "as a result of seeing 
today's presentation, my knowledge of the above subject has 
increased," and then provided a four-category Likert scale 
for a single response: "Not at all," "A little," "Some," or 
"Quite a bit." These survey questions are more specific than 
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TABLE II: Educational attainment of survey participants. 



Students 

Lifelong Learners 

Educators 

Not yet completed high school 

98% 

7% 

0% 

High school graduate or GED 

2% 

9% 

5% 

Some college, no degree 


12% 

7% 

Associate's degree 


16% 

5% 

Bachelor's degree 


35% 

26% 

Graduate/advanced degree 


21% 

58% 


the "Did you learn anything new"? (yes/no) in the study by 
Goldman et al. (2010), because we were targeting perceived 
learning of specific literacy principles. We provided the same 
four-category scale for the engagement question, "After 
today's presentation, how excited are you about the great 
unexplored deep ocean frontier and the exploration and 
research of deep-sea vents"? Educators received an addi¬ 
tional question for the likelihood of using what they learned 
in their educational activities (four-category scale from 
"Definitely won't" to "Definitely will"). 

To attract participation in the survey, we hosted three 
events for the public, including two free family science nights 
at the Ocean Explorium, highlighting the LWS or the SFU 
narrative. One reason for holding events at no cost was to 
attract visitors of lower socioeconomic status to the museum; 
previous research has indicated that lower socioeconomic 
status correlates to lower knowledge of environmental and 
ocean sciences (e.g.. Steel et al., 2005). We also highlighted 
the new content during school field trips and a teacher 
professional development workshop at the Ocean Explo¬ 
rium. For the workshop, we focused on attracting teachers 
from schools in the Massachusetts South Coast region. Of 
the total of 158 survey responses received, 135, or 85%, 
included data for self-reporting of knowledge gained and 
level of engagement. Although we attracted many visitors to 
the museum during the two free events and we offered 
incentives to those filling out the surveys, it was challenging 
to obtain completed surveys (e.g., 18 completed surveys for 
160 visitors, or 11%, at the second free event). 

To assess perceived learning outcomes and excitement 
generated by viewing the presentations, we examined 
descriptive statistics (i.e., histogram, mode, and median of 
responses) and conducted nonparametric statistical tests 
appropriate for the categorical data, assigning the progres¬ 
sive positive integer values 1 through 4 to our four-level 
Likert scale. We did not infer the distance between each 
successive item category to be equivalent and thus only 
performed tests with ranks. Within each group (students, 
lifelong learners, and educators), we tested the effectiveness 
of the movie vs. the docent-led presentation using a 
Wilcoxan rank sum test, and we tested for differences 
among the groups for each type of presentation using a 
Kruskal-Wallis test (e.g., Hollingsworth et al., 2011). Our 
numbers of responses led to an unbalanced design (Supp. 
File 2, which can be found online at http://dx.doi.org/10. 
5408/14-067sl), precluding a Friedman test as a nonpara¬ 
metric two-way analysis of variance. All statistical tests were 
performed with Matlab software R2010b (The MathWorks, 
Inc). 


STUDY DEMOGRAPHICS 

Our total of 135 participants included 48 students and 87 
adults (43 lifelong learners and 44 educators). For the 
students, most (81%) resided in the Massachusetts South 
Coast region (New Bedford 29%, Fall River 17%, and 12 
surrounding towns 35%; Table I). For the level of educa¬ 
tional attainment, almost all students selected "Not yet 
completed high school" (Table II). Half of the students were 
exposed to either narrative, and for each narrative approx¬ 
imately half saw the movie while the other half saw the 
docent-led presentation (see Supp. File 2 for maximum 
number of respondents to each survey question). 

A similar majority (85%) of the adult lifelong learners 
came from the Massachusetts South Coast region (New 
Bedford 40%, Fall River 5%, and other South Coast towns 
40%; Table I). Their levels of educational attainment 
represented all six categories, including "Not yet completed 
high school" (7%; Table II); however, the percentage with 
bachelor's degrees (35%) was greater than expected for the 
South Coast region and more aligned with the state of 
Massachusetts. Half of the lifelong learners were exposed to 
either narrative, and for each narrative approximately half 
saw the movie while the other half saw the docent-led 
presentation (see Supp. File 2 for maximum number of 
respondents to each survey question). 

For the adult educators who participated in the survey, 
67% were from the Massachusetts South Coast region (New 
Bedford 28%, Fall River 9%, and other South Coast towns 
30%; Table I). Their level of educational attainment exceeded 
the lifelong learners, with more than half (58%) of the 
educators having graduate degrees (Table II). More educa¬ 
tors saw the LWS narrative; our sample size was least for the 
educators who saw the docent-led presentation of SFU 
(Supp. File 2). 

RESULTS FOR ASSESSMENT OF LEARNING 
AND ENGAGEMENT 

For all combined survey respondents, both the LWS and 
the SFU narratives led to median responses of "Some" to 
"Quite a bit" of perceived learning per literacy principle 
(Table III). Comparing the two narratives for all combined 
respondents, self-reported knowledge gains ranged from 
91% to 94% per literacy principle (inclusive of the "A little," 
"Some," and "Quite a bit" categories) for LWS and 94% to 
98% for SFU (Supp. File 2). Comparing descriptive statistics 
(histograms, modes, and medians) between the two narra¬ 
tives, LWS (Fig. 2) led to greater perceived learning for the 
biology principle, while SFU (Fig. 3) led to greater perceived 
learning for the Earth Science principle (Table III, Supp. File 












TABLE III: Medians of survey responses. "Quite a bit" responses are in bold. 
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2). Both narratives were equally effective in generating 
"Quite a bit" of excitement (Fig. 4 and Table III). Results of 
the Wilcoxan rank sum tests indicated no significant differ¬ 
ences between the type of presentation (movie vs. docent 
led) for the respondents grouped as students, lifelong 
learners, or educators (Supp. File 2). Results of the 
Kruskal-Wallis tests also indicated no significant differences 
among students, lifelong learners, and educators for either of 
the movies or the docent-led presentations (Supp. File 2). 
However, some interesting patterns for learning outcomes 
are apparent when comparing the descriptive statistics 
within and among the groups of respondents, which we 
highlight below. 

Assessment of Students 

Based on median responses, students appeared to learn 
more from LWS than from SFU (Table III), although this 
distinction was not apparent based on modes or combined 
percentages of self-reported new learning per literacy 
principle (Supp. File 2). For LWS, histograms for student 
learning outcomes were well matched between the two 
types of presentations [Figs. 2(a), 2(d), and 2(g)], especially 
for the biology and Earth Science principles. For the 
exploration principle, students were split between reporting 
either "A little" or "Quite a bit" of knowledge gained, with 
greater median response for the movie [Fig. 2(g) and Table 
III]. For SFU, histograms for student learning outcomes also 
appeared well matched between types of presentation [Figs. 
3(a), 3(d), and 3(g)], although the median responses suggest 
more perceived learning from the docent-led presentation 
for the Earth Science and exploration principles (Table III). In 
terms of excitement about the great unexplored deep ocean 
frontier, students responded more to the docent-led 
presentation of LWS [Fig. 4(a)], but their response was 
most dramatic (100% "Quite a bit") to the movie of SFU 
[Fig. 4(d)]. 

Assessment of Lifelong Learners 

In general, lifelong learners reported similar to slightly 
more knowledge gains than students for both narratives 
(Table III, Supp. File 2). Based on median and mode 
responses, neither narrative appeared more effective overall 
for lifelong learners; however, for LWS, 100% of the lifelong 
learners reported at least "A little" new learning for all three 
literacy principles for both types of presentations [Figs. 2(b), 
2(e), and 2(h)]. For LWS, histograms for learning outcomes 
for lifelong learners were not well matched for the Earth 
Science and exploration principles, with the movie appearing 
more effective than the docent-led presentation [Figs. 2(e) 
and 2(h)]. For SFU, lifelong learners had greater response to 
the docent-led presentation for all three literacy principles 
[Figs. 3(b), 3(e), and 3(h) and Table III]. In contrast to the 
students, for level of engagement, the lifelong learners 
responded more to the movie of LWS [Fig. 4(b)], but most 
dramatically (100% "Quite a bit") to the docent-led 
presentation of SFU [Fig. 4(e)]. 

Assessment of Educators 

In general, responses to literacy principles indicated that 
educators perceived less learning than lifelong learners for 
both narratives (Table III, Supp. File 2), and this was 
particularly evident for LWS (Fig. 2). Overall, educators 
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FIGURE 2: LWS narrative, perceived learning outcomes for three scientific literacy principles: upper (a-c), biology; 
middle (d-f). Earth Science; lower (g-i), exploration. Y-axis = percentage of respondents. (Left) Students (number of 
respondents, n — 14 movie, 9 docent-led); (center) lifelong learners (n = 12 movie, 9 docent-led), and (right) educators 
[n — 12 movie, 14 docent-led, with the exception of 13 docent-led in (f)]. 


appeared to learn more from SFU (Table III, Supp. File 2). 
For LWS, histograms for educator learning outcomes were 
well matched between the two types of presentations [Figs. 
2(c), 2(f), and 2(i)], which was not the case for SFU [Figs. 
3(c), 3(f), and 3(i)]. Educators responded more to the docent- 
led presentation of SFU for the biology and Earth Science 
principles, which was similar to the lifelong learners, but 
more to the movie for the exploration principle, in contrast 
to lifelong learners (Fig. 3, Supp. File 2). Educators were not 
as excited as students or lifelong learners by either 
presentation of LWS [Fig. 4(c)]. However, the SFU movie 
led to the most dramatic (100% "Quite a bit") excitement for 
educators [Fig. 4(f)], matching the students. Combining 
responses to both narratives and both types of presentation, 
most educators reported that they probably or definitely 
would incorporate their new learning into their classrooms 
(92% combined responses; Fig. 5). In particular, for the 
docent-led presentation of SFU, 100% of the educators 
reported that they definitely would incorporate their new 
learning in the classroom [Fig. 5(b)]. 


DISCUSSION 

Digital Globes Effective in Teaching OLPs and ESLPs 

Our survey results support a key finding from the cross¬ 
site summative evaluation of SOS as a platform for informal 
education—that visitors report new learning from SOS 
presentations (Goldman et al., 2010). Our survey results 
demonstrate more specifically that students and adults 
perceived an increase in their knowledge of ESLPs and 
OLPs presented through our new content developed for 
SOS. Our results for self-reported knowledge gains ranged 
from 91% to 98% per literacy principle for all combined 
respondents, compared to 71% of visitors reporting new 
learning in the SOS cross-site summative evaluation (Gold¬ 
man et al., 2010). In general, perceived learning outcomes in 
our study appeared greatest for the adult lifelong learners, 
followed closely by students; adult educators generally 
reported less learning, as expected with their higher level 
of educational background. Also not surprising was our 
finding that each narrative led to greater learning for the 
particular disciplinary principle reflected in its respective 
title. Our attempts to increase the participation of under- 
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FIGURE 3: SFU narrative, perceived learning outcomes for three scientific literacy principles: upper (a-c), biology; 
middle (d-f). Earth Science; lower (g-i), exploration. Y-axis = percentage of respondents. (Left) Students (number of 
respondents, n — 9 movie, 10 docent-led), (center) lifelong learners [n — 10 movie, 9 docent-led, with the exception of 
8 docent-led in (h)], and (right) educators (n — 6 movie, 4 docent-led). 


represented populations may have led to an enhancement of 
perceived learning outcomes, given previous research 
correlating knowledge to socioeconomic status (Steel et al., 
2005). 

We did not find a significant difference between the 
effectiveness of the movie and the docent-led presentation 
for any of the literacy principles, but this may be expected 
because we tried to closely match the content and the script 
within the presentations. The SOS cross-site summative 
evaluation tested presentations categorized as facilitated, 
auto-run, or visitor-initiated and found that facilitation 
correlated with learning from SOS presentations (Goldman 
et al., 2010). However, in our study, we tested narrated 
compilation movies as another type of presentation, which 
differs from auto-run of datasets in the SOS Data Catalog; 
our movies tied together the datasets using narration, 
graphic art, additional imagery, and portions of the site- 
specific movies. Also, because we tried to closely match our 
compilation movies and docent-led presentations, our live 
shows were not as facilitated as they would be if tailored to 
specific audiences. Our results are more in line with a recent 


study using the SOS to teach climate literacy that found that 
short (5 min or less) narrated movies delivered within live 
shows yielded "a slight but not substantial advantage" over 
presentation of the same movie within an auto-run show (p. 
485 in Schollaert Uz et al., 2014). Our study differs from 
Schollaert Uz et al. (2014) in that our live, docent-led 
presentations are intended to step more deliberately through 
the same datasets that are incorporated into our compilation 
movies. 

Alternatively, it is possible that a statistical difference 
may be detected between our movie and our docent-led 
presentations with greater sample sizes and/or the use of 
more categories of responses. We might expect more 
learning from the docent-led presentation because the 
audience had more time to interact with the materials (20 
min, as opposed to 5-min movies). Schollaert Uz et al. (2014) 
suggested that an advantage in learning from live shows 
could partly be because of slower pace and ability to repeat 
concepts. Within each group of respondents in our study 
(students, lifelong learners, and educators), some differences 
between the two types of presentations are apparent in the 
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FIGURE 4: Excitement stimulated by SOS narratives: upper (a-c), LWS; lower (d-f), SFU. Y-axis = percentage of 
respondents; n — number of respondents. (Left) students, (center) lifelong learners, and (right) educators. 


histograms, modes, and medians of responses to particular 
literacy principles in either narrative. However, no clear 
differences between the two types of presentations emerge 
for either narrative when considering all three literacy 
principles within a group (perhaps with the exception of 
lifelong learners responding to the docent-led presentation 
of SFU) or when considering a single literacy principle across 
groups. 

Narrated compilation movies are a relatively new format 
for the SOS (Starobin, 2006), and we recommend additional 
research to better understand how to design movies to teach 
science with digital globes. When the cross-site summative 
evaluation was completed in 2010, only about a dozen 
narrated movies with SOS datasets were available, and as of 
summer 2015, about six times as many were available 
(NOAA, 2015). We think there are advantages when 
developing content for digital globes to produce a compi¬ 
lation movie, in addition to providing datasets and 
visualizations separately for facilitated, auto-run, and visi¬ 
tor-initiated presentations. An advantage of a movie is that 
the same carefully constructed information is delivered every 
time. A movie can aid docent training. There is inherent 
variability in a docent-led presentation in that the delivery 
depends on the individual docent's training, knowledge, and 
style and on the audience's interaction (e.g., questions from 
the audience). That said, it can be an advantage of the 
docent-led presentation for delivery to be tailored to the 
audience (e.g., repeat information and address questions 
from the audience). Also, a docent-led presentation can 
change over time to incorporate new scientific findings, 
while the movie may become outdated. 


We recommend additional evaluation to compare 
learning of Earth system science that results from different 
spherical display systems, including virtual globes, and 
whether spherical technologies enhance retention of certain 
literacy principles. A key finding from the SOS cross-site 
evaluation was that "Visitors feel seeing information on the 
sphere is more realistic and provides more perspective" (p. 2 
in Goldman et al., 2010). Schollaert Uz et al. (2014) found in 
free-response comments from SOS viewers that "seeing the 
huge Earth" and found that a school group retained learning 
4 d after seeing a live SOS show or movie. Our compilation 
movies are available for other spherical display systems, 
including Magic Planet (http://globalimagination.com/) and 
iGlobe (http://www.iglobeinc.com). These digital globes are 
made in a variety of sizes, which may affect the audience's 
experience with the materials and consequent perceived or 
actual learning, as well as retention of learning. Digital 
globes are also being used in formal classroom settings and, 
to the best of our knowledge, have not been evaluated yet as 
part of a formal science curriculum. 

As digital globes are increasingly being used in informal 
and formal education, we recommend more rigorous 
evaluation using pre- and posttest experimental designs to 
better understand the extent to which digital globes are 
effective in teaching Earth system science. Our posttest-only 
study was restricted to perceived learning outcomes; 
however, a pre- and posttest design with questions to test 
for correct responses before and after viewing content on the 
globe would allow quantification of actual learning to better 
assess the impact category "Awareness, knowledge or 
understanding (of)" (Friedman, 2008). A second posttest 
delivered some time later, for example, as conducted by 
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FIGURE 5: Educator responses to "Will you use what 
you learned today in your educational activities"? (a) 
LWS. (b) SFU. Y-axis = percentage of respondents; n — 
number of respondents. 


Schollaert Uz et al. (2014), would quantify retention of 
learning from the digital globe. 

Digital Globes Effective in Generating Excitement 
About the Deep Ocean Frontier 

Our two narratives clearly generated excitement about 
the great unexplored deep ocean frontier and the exploration 
and research of deep-sea vents, and we attribute this success 
to the tight collaboration among scientists, educators, and 
graphic artists in developing the content for public audiences. 


In particular, we propose that the collaboration enhanced the 
storytelling and the visual impact of our materials. As 
explained on p. 14 in Ma et al. (2012b), scientists "cannot 
easily develop and produce high-quality visualizations" 
because of factors that include "lack of expertise in the fields 
of art, visualization, and storytelling." For example, in 
developing the narratives, we considered "narrative appetite" 
in which the audience "must want to know what happened" 
to continue to be engaged with the sphere presentation (p. 
541 in Norris et al., 2005). We also considered carefully our 
audience as "interpreting agents," with previous research 
indicating that individuals construct their own meaning when 
interpreting a narrative by "inferring, construing, projecting, 
...anticipating" (p. 544 in Norris et al., 2005). In terms of 
choices made for visual impact, for example, initially we 
attempted to display imagery from deep-sea vents in a 
geographic context: in a rectangular image frame centered on 
the position on Earth where the imagery was collected. 
Ultimately, we decided instead to display the imagery on an 
entire hemisphere, which had the effect of looking at the 
scene as if in a fishbowl. The effect is similar to how the scene 
appears to human eyes through the thick viewports of deep- 
diving human-occupied vehicles. 

Similar to our findings for the literacy principles, no clear 
differences between the movie and the docent-led presen¬ 
tation emerged for excitement when considering within a 
group or across groups. For example, the students were 
more excited by the docent-led presentation of LWS but 
clearly were more excited by the movie for SFU—and vice 
versa for the lifelong learners. Our results, however, are 
specific to the 6-ft-diameter SOS spherical display; to 
determine in general if digital globes are effective in exciting 
public audiences, we recommend evaluating content dis¬ 
played on other, smaller systems. 

Broader Impacts of Our New Content for SOS 

The "95% solution," as coined by Falk and Dierking 
(2010), suggests that most STEM learning occurs in informal 
educational settings. Free-choice, science learning institu¬ 
tions have an important role in "promoting lifelong learning 
and nurturing environmental and ocean literacy" (p. 126 in 
Schubel et al., 2009). By contributing our new datasets, 
movies, playlists, and scripts to the SOS Data Catalog, we 
have the capability of providing our materials, tuned to 
ESLPs and OLPs, to more than 100 SOS facilities in 20 
countries, reaching up to 33 million people per year (NOAA, 
2014). In addition, our movies are available freely from an 
open-access server for other spherical display platforms and 
can reach many more people, especially as digital globes are 
incorporated into more settings—both informal and formal 
(Vanhoenacker, 2013). 

In this project, we focused on reaching underrepresent¬ 
ed students, and teachers of such students, in the 
Massachusetts South Coast region. We are hopeful that we 
not only educated and excited the students who participated 
in our study but also inspired a few next-generation ocean 
explorers and scientists. With teachers reporting that they 
are likely to incorporate their new learning into their 
classrooms, we are hopeful to extend the impact of the 
SOS presentations. We offered two free family nights to 
attract people who otherwise might not be interested in or 
able to afford a visit to a science museum; however, our 
future plans include adapting some of our materials for 
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virtual globes to be able to reach people outside of the 
museum setting (e.g., Blaschke et al., 2012). 
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